The present invention relates to video signal processing, and more particularly to depth-based video combining which creates a combined depth that preserves smoothness for further combining operations.
Digital picture manipulation devices take an incoming video image and remap the pixels to another video image to make the incoming video image appear to have been moved or warped in three dimensional space. A digital picture manipulator (DPM) transforms one image per channel, with this image containing luminance, chrominance and key (opacity) information. Often a digital picture manipulator manipulates more than one video channel and the channels are combined into a single resulting channel. A problem arises when a pixel of the combined image has contributions from two transformed source images, i.e., a decision has to be made to give priority to one video image over the other. The Kaleidoscope DPM-1 digital picture manipulator, manufactured by The Grass Valley Group, Inc. of Nevada City, Calif., United States of America, uses the calculated depth of the transformed source images to determine priority, i.e., the image appearing closer to the viewer having priority over the farther appearing image.
The Kaleidoscope digital picture manipulator's depth-based combiner creates a priority signal that determines the priority of a foreground image over a background image. This priority signal is input to a combiner as described in U.S. Pat. No. 4,851,912 issued Jul. 25, 1989 to Richard A. Jackson and Kevin D. Windrem entitled "Apparatus for Combining Video Signals." Which signal is the "foreground" and which the "background" is determined by depth, and can switch on a pixel-by-pixel basis for a combined image, as in the case of intersecting planes. The Kaleidoscope DPM creates the priority signal by using depth information from each image. This information is called W, the depth coefficient, where EQU 1/W=Z/V.sub.p +1
V.sub.p being the simulated distance of the viewer from the plane upon which the images are perspectively projected and Z being the distance from that plane to the transformed image in three dimensional space prior to perspective projection. W gets smaller, decreasing to zero, as an object in the image goes to infinity, and gets larger as the object in the image approaches the viewer, being clipped at a value of 8 in the Kaleidoscope DPM. Priority is formed from each image's W by the following function, commonly called a clip and gain function, EQU P=CLIP(GAIN.times.(W.sub.A -W.sub.B))+1/2
where W.sub.A is the depth coefficient for image A, W.sub.B is the depth coefficient for image B, GAIN is a softness multiplier and CLIP is a function that returns -1/2 for values less than -1/2 and returns +1/2 for values greater than +1/2 and passes all values inbetween. The output of the circuit is a number ranging from zero to one. When P is less than 1/2, image A is behind image B and image B has more priority, and when P is greater than 1/2, image A is in front of image B and has more priority. Priorities between zero and one give smoothness to intersecting images and are more visually pleasing than sharp intersections where only zero and one are possible.
A problem associated with depth combining involves a situation where there are more than two images competing for priority. The Kaleidoscope DPM uses a serial method for combining where two images are combined to form a resultant image which is then in turn combined with a third image, whose result is combined with a fourth image, etc. This requires a series of two-input video combiners. Each of the source images has a depth signal, as does each successively combined image. With the serial combiner approach the Kaleidoscope DPM selects the nearest of the two depth values to determine the depth of each combined image. However when the nearest image is nearly transparent, such as along soft edges, setting the resulting depth to that of the nearest image gives the combined image a stronger depth when mixing with a third image later on. For example during a soft edge the depth is that of the near image, even though some of the far image may be seen, and then where the foreground image fades completely, the depth abruptly jumps to that of the far image, i.e., the depth component of the combined image has abrupt changes. If a third image is combined with this result, the abrupt change in depths may cause an abrupt change in the priority signal used in the next combining operation. Such abrupt changes give unpleasant harshness to video images. It would be more pleasing if the depth smoothly changed from that of the nearest image to that of the farthest image during the soft edge.